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You have asked me to talk about some of the problems and possible 
solutions associated with the evaluation of projects to be conducted 
under Title 1 of the Education Act* 2 I am intensely interested in this 
tjpic and am currently devoting most of my efforts to it* The more I 
learn about the Title I program, the more I am impressed by the wide range 
of difficult evaluation questions which educators are asking and by the 

lack of practical answers that are available* Hopefully, during this 

\ 

conference we can tackle some of these tough questions and move toward 
reaching some viable, operational- level solutions* 

Each of you, I am certain, Is acutely aware of the urgent need to do 
this* Under Title I alone, over one billion dollars is authorized for 
fiscal year 1966* While this is an unprecedented opportunity to improve 
education, it also brings a challenge; the schools must provide for 
effective evaluations* That is, they must present objective data which 
demonstrate the effectiveness of their new plans and programs* 
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The need for effective evaluations takes on increased importance 
when one considers the implications of such evaluations* At the Federal 
level, allocations for succeeding years will be largely dependent upon 
evaluation results for programs conducted during these early years* At 
the local level, decisions about retaining, abandoning, or revising pro- 
grams can be made more Judiciously if sound, objective data are available* 
It is obvious that solutions to the evaluation problem must be found 
Immediately* Further, such solutions must be tailored to the intent and 
characteristics of the Education Act* 

NATURE OF THE EDUCATION ACT 

The purpose of the Education Act is to meet the special educational 
needs of deprived children* Somewhat more specifically, it is to in- 
crease the educational attainment, experiences, and opportunities of 
these children* The program is national in scope, design, support and 
broad control* It is to be coordinated and specifically controlled at 
the state level and to be implemented in the local school districts* The 
over-riding, long-range purpose, as I see it, is to promote planned change 
in education on a continuing basis, that is, to close the gap between 
theory and practice* 

To achieve this broad purpose, many changes obviously will be re- 
quired across the face of education* New methods and materials of many 
kinds will have to be developed* These will have to be publicized and 
demonstrated to the schools which need such innovations* The local 
schools will have to adopt and adapt those innovations which are best 
fitted to their needs* Thus, a wide range of activities are appropriate 
under the Act* Further, these will need to be coordinated into a program 
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of planned change in education, if the impact of the Title 1 program Is 
to have far-reaching implications for meeting the needs of disadvantaged 
children, and if wasteful duplications of effort are to be avoided, 

A Scheme for Classifying Education Act Projects 

David L, Clark and Egon G, Guba^ have proposed a taxonomy for 
classifying the many activities that may be sanctioned under the Act, 
Originally, they developed this taxonomy to cover the range of activities 
necessary for producing planned change in education, I do not wish to 
enter a discussion of change process research and theory per se, but I 
do want, from time to time this morning, to refer to the Clark-Guba 
taxonomy in order to illustrate the process and function of evaluation 
within the total framework of change and innovation, I should, therefore, 
now like to give you a brief introduction to this taxonomy. 

Figure 1 contains a modified version of the Clark-Guba taxonomy, 

(Any criticisms that you may wish to make of this taxonomy should be 
directed toward me, since I have adapted it to fit my purposes. But, 
any credit should go to Drs, Guba and Clark since they master-minded the 
development of the original scheme). 

You will note that 1 have converted this taxonomy into the form of 
a flow diagram, the flow being from top to bottom. The change process as 
depicted here includes the activities of research, development, diffusion, 
adoption, and evaluation, Research is depicted by a triangle, development 

David L, Clark and Egon G, Guba, An Examination of Potential 
Change Roles In Education , Paper presented before the Seminar on 
Innovation in Planning School Curricula, October, 1$65« 




*Based upon “A Classification Scheme of Processes 
Change in Education, 11 by David L, Clark and Egon 
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by convex hexagons, diffusion by concave hexagons, adoption by rectangles, 
and evaluation by diamonds* Each activity (with the exception of 
evaluation) is classified, from left to right, by the agencies which may 
be expected to be primarily involved in that activity, by the objective 
associated with the activity, by some of the criteria associated with 
the objective, and by the relation of the activity to the change process* 

1 shall be concerned primarily with the objectives and criteria associated 
with each activity and with the overall relation of evaluation activities 
to the change process. Before discussing this chart further, I now want 
to move to a consideration of the nature of evaluation in the Tlt<e I 
program* 

EVALUATION DEFINED 

The guidelines for Title I define evaluation as the process of 
determining the extent to which objectives have been reached* To 
evaluate, it is therefore necessary to know what the objectives are and 
to specify criteria for determining the relation of the outcomes to the 
objectives* Since different kinds of activities have different objectives, 
different kinds of evaluations may be distinguished. 

From Figure 1 you will note that an evaluation activity immediately 
follows each of the other change activities* This is intended to depict 
that evaluation is appropriate at each stage of the change process and 
that different kinds of evaluations may be distinguished for different 

levels of objectives and criteria. 

For example, in test development, evaluations of test plans differ 
from evaluations of test items* The objective associated with developing 
a test plan is to des 1 gn or blueprint the test to be developed. The 
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criteria for evaluating a test plan are content validity , feasibility , 
and estimated reliability . The usual method of evaluating a test plan 
is to secure expert judgment , e.g., from curriculum and measurement 
specialists* A method quite different from this one would be employed 
in evaluating the actual test questions once they were developed. The 
objective in developing test questions is to construct questions which 
meet the design specifications. The criteria that are appropriate for 
evaluating test questions are design specifications (i«e., do the 
questions cover the specified content and objective areas?) and i nd i - 
viduai performance (i.e., do the questions possess concurrent validity 
and reliability?). The method usually employed in evaluating test 
questions is to administer them and certain criteria instruments to a 
sample of students and then to perform item analyses . 

We have seen from this example that different approaches to 
evaluation were required, due to different objectives and criteria for 
the two activities. In the first case, expert judgment was employed; 
in the second, statistical analyses were used. This example could be 
extended to other cases and other phases of the diagram, but in the 
interest of time t shall not attempt this. I think the point is clear: 
evaluations may differ widely as a function of objectives and criteria* 

I should now like to move to a consideration of the role of evaluation 
under Title I. 

THE ROLE OF EVALUATION UNDER TITLE I 
As I see it, the purpose of evaluation in the Title I program is 
to assist in making decisions. Based on evaluative data, decisions will 
be made to continue, discontinue, or modify projects and practices. 
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Further, such decisions will be made at three administrative levels: 
Federal, state and local. Evaluations must, therefore, be designed to 
provide the data that will be required for making such decisions. 

Returning to Figure 1 once again, you will note that arrows lead 
from each evaluation activity to both the preceding change activity and 
the succeeding change activity. This is intended to depict the decision 
function of evaluation. At each stage in the change process, outcomes 
are compared with objectives on the basis of established criteria. If 
the criteria have been met, the process flow continues to the next 
activity; if not, the flow reverts to a prior activity in order that 
corrections can be made. I realize that reality will not often conform 
to this neat diagram, that in the change process many of the depicted 
change phases may often be skipped, and that the alternative of 
"scrubbing 11 a mission (to borrow a phrase from the space industry) is 
always present at any phase of the change process, for any particular 
activity. Although Figure 1 does not adequately account for all such 
alternatives I think it does depict the general function of evaluation. 
This function, as I see it, is to insure sound, systematic progress 
toward putting theory into practice and to servo as a means for identi- 
fying problems and taking corrective action as each failure is 
encountered. A further function, not so well depicted by this chart, 
is an information feedback function. Evaluations can continually pro- 
vide information pertinent to the conduct of other projects. For 
example, evaluative information about a demonstration project might 
well be useful to a researcher who is studying educational change. 
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It is important to note that decisions about Title I practices 
will be made at Federal , state, and local levels, while the bulk of 
basic data for making these decisions will be collected at the local 
level alone. Thus the evaluation designs of local schools must be 
consistent with state and Federal requirements for evaluation. 

Obviously, then, Federal and state objectives and criteria must be 
specified and, further, specific evaluation designs must be developed 
for each of these levels. Only after this has been accomplished can 
local schools prepare evaluation designs which will provide the required 
data. 

I should now like to suggest a scheme for viewing in greater 
detail the decision process as it may occur in the Title 1 program. 

A CONCEPTUAL SCHEME FOR VIEWING THE DECISION PROCESS 

Please direct your attention to Figure 2. This contains a feed- 
back control loop which is intended to delineate the evaluation loops 
depicted in Figure 1. The specific intent is to clarify the decision 
function required by Title I. The inner loop depicts the local school 
activities; the intermediate loop, the state activities; and the outer 
loop, the Federal activities. 

Starting at block 1, we have the initial locai school project 
operations. Proceeding to the right, data collection is depicted by 
the first segment of the curved line. This occurs at the local school 
level and is seen to be purely objective. Here, no judgments are in- 
dicated or appropriate. What is indicated is the systematic collection 
of all the data that will be needed for later decisions at local, state, 
and Federal levels. Thus, it would be useful to know at this stage what 
kinds of decisions will need to be made at each of these levels. 
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It iMn< to mo that basically two kinds of decisions will be 
required. At the state and Federal leve1s 9 the basic decisions will 
be to increase, decrease, or discontinue entirely the funding of various 
kinds of projects. At the local level, the most prevalent and signifi- 
cant decisions, In my opinion, will be whether to continue or modify 
methods and materials as a project proceeds, i.e,, as more is learned 
about an on-going project. 

Product Evaluations 

For the state and Federal decisions, product or outcome data will 
be needed. Here, the traditional kind of product evaluation is appro- 
priate. By product evaluation I mean the process of relating outcomes 
to goals on the basis of specified criteria . This is the kind of 
evaluation that the majority of the material on evaluation in the Title 
I guidelines is concerned with. I shall not discuss the details of 
product evaluation here since they are we 1 1 -discussed elsewhere. But 
I do want to make the following assertions about product evaluations 
as they relate to Title I: 

(1) Product evaluations arc usually in the form of 
a comparison of pre-project and post-project 
performance. 

(2) The best that such evaluations can do is 
to indicate whether the projoct outcomes were 
higher, the same as, or lower than some standard. 
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(3) Product evaluations provide the kinds of 
information that will be needed for decision- 
making at the state and Federal levels, but 
are not adequate for assisting in the decision 
process at the local level. 

(4) To design effective product evaluations, 
it is necessary to know what the objectives 
and associated criteria are for the Federal, 
state, and local levels. 

(5) For the evaluative data to be comparable 
for similar projects, It will be necessary 
that product evaluation plans be specified 

at both the Federal and state levels. 

As noted, product evaluations do not provide the information that 
is needed for making decisions at the local level. Product evaluative 
data usually will be available only annually, and then will contain 
only the gross information about the effectiveness of an overall pro- 
ject. Such data will be available too infrequently and will lack the 
specificity required for adequate project control and direction at the 
local level. 

Process Evaluations 

For local control and decisions, i believe that a process type of 
evaluation is required. Process evaluation, as 1 define it. is the 
method of relating process to product . The overall strategy Is to 
identify and monitor, on a continuous, molar, non- interventionist basis 
the potential sources of failure in the project, e.g., people, 
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facilities, equipment, and materials, Roger Barker* has recently 
characterized this approach as transducer type Inquiry In an ecological 
context. Here, the evaluator does not exercise experimental control 
over the situation, nor does he manipulate the situation In any way. 
Rather, he accepts the situation as It Is and as It evolves under real 
life circumstances. He then "bugs" the situation as best he can by 
focusing his best observation and other non- Interventionist data 
collection techniques on those aspects of the project which are most 
crucial to Its success. The nature of such an evaluation Is multi- 
variate, and not all of the Important variates can be specified prior 
to the Initiation of a project. The process evaluator consequently 
focuses on those variates which are theoretically Important, but also 
remains alert to any unanticipated, but significant events that may 
occur along the way. In summary, under process evaluations, data are 
collected day-by-day, organized systematically, analyzed periodically 
(e.g., weekly), and reported as often as the project director may 
require such information (e,g«, monthly), 

I should now like to direct your attention to block number 2 of 
Figure 2: the organization of data. Here, data would be coded, pro- 

cessed (e,g,, keypunched) and filed. This would be necessarily a day- 
by-day process. 



Stager 6, Barker, "Explorations In Ecological Psychology," 
American Psychologist, 20 (1965), 1-14. 




13 



As the stats and local reports ware required, the data, both 
process and product collected at block 2, would be analyzed (at 
block 3) In accordance with the requirements of these reports# The 
process data would be analyzed frequently to determine whether the 
potential sources of failure were becoming actual failures, whether 
the procedures were following the project design, whether the projects 
were on schedule, and whether the design or procedures should be 
modified. Once the outcomes were determined, the process data could 
also be used to diagnose the outcomes# The product data, on the other 
hand, would be analyzed Infrequently# Essentially, this type of 
evaluation would determine the overall effectiveness of the project 
In meeting u»e prescribed, general goals. 

The evaluator would send process evaluation reports to the project 
director quite frequently# On the basis of these reports, the project 
director would make decisions (at block 4) about a project# These 
decisions would be implemented (at block 5) and, as a consequence, an 
on-going project would undergo frequent modification (block l)# As I 
view It, this cycle Is continuous, and the frequency of process reports 
Is limited only by the frequency with which they are required by the 
project director. The purpose of these process reports is obviously to 

assist In project control and evolution# 

Returning to block 3 t product evaluations would be prepared at 
least annually and sent to the state education department# The state 
education department would, of course, receive such reports from all 
public school districts In the state. The immediate task of the state 
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•ducat ion department would, therefore, be to organize the report* Into 
type* of project* and to combine the data for similar project* (block 6). 
Next, thl* organized data would be analyzed (at block 7) to determine 
the itrengths and weaknes.e* of the .tate-wlde program. Bated on the*e 
re*u1t», the *tate coordinator for Title I program* would make decisions 
(at block 8) about program empha*e* and state control of the program. 
These decision* would be Implemented (at block 9) and would thus affect 
the state program operation* (block 10). Of course, thl* would In turn 
affect the local school project operations (back at block 1). 

Returning to block 7, you will note that annual evaluative reports 
from the fifty states would be sent to the Federal agency. The Federal 
agency would thus be faced with organizing the data from the fifty 
states (at block 11) so that major program thrusts could bo examined, 
e.g., those of educational achievement, opportunities, and experiences. 
These data would then be analyzed (at block 12) on a nationwide basis 
and the results reported to the Congress and the President. Decisions 
(block 13) would then be made at the Federal level about program 
emphases and funding. The Implementation (at block 14) of such 
decisions would affect the operations of the Federal program (block 15). 
Then the operations of state programs (at block 10) and the local school 
project (at block 1) would correspondingly be affected. 

I have now completed a somewhat rambling tour of Figure 2. Hope- 
fully, I have convinced you of the need for developing and coordinating 
evaluation plans at Federal, state and local levels. Also, I have tried 
to Indicate that data for all three of these types of evaluations will 
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be largely collected at the local level, end that these date will form 
the basis for Federal, state, end local decisions, which will effect 
local operations. Finally, I have suggested that while product 
evaluations will suffice for state and Federal decisions, process 
evaluations are required for effective project control and evolution at 
the local level. I hope that it is clear that evaluation can be e very 
positive force for improving education, and that, in fact, educational 
progress is largely dependent upon effective evaluation. I should now 
like to move to a discussion of some of the problems associated with 
fulfilling the role for evaluation that 1 have just described. 

PROBLEMS IN FULFILLING THE ROLE FOR EVALUATION IN THE 
TITLE 1 PROGRAM OF E.S.E.A. 

In my opinion, there are three basic problems standing in the way 
of implementing the role for evaluation as outlined above. These are: 

(1) a lack of evaluation experience In education, (2) a lack of 
trained evaluators, and (3) a lack of non-interventionist type 
evaluation designs. I doubt that anyone would refute that these are 
important evaluation problems. The current question, then, is how to 
find immediate solutions to these problems, so that evaluation require- 
ments for the Title I program can be effectively met. In Ohio, it was 
clear that no one kind of agency, l«e., schools, universities, or the 
state education department, possessed the resources necessary for solving 
these problems. It was further clear, that these agencies could fair 
better by working together than by attacking the evaluation problems 
independently. 
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In this regard, I am going to proceed by describing the general 
approach that our Evaluation Center at The Ohio State University is 
taking in evaluating Title I projects* Then I will close by making 
several recommendations regarding evaluation* 

APPROACH TO EVALUATION OF TITLE I PROJECTS 
AT THE OHIO STATE UNIVERSITY 

The Evaluation Center at The Ohio State University was established 
last July in response to the needs that were then oeing posed by the 
Education Act. The purposes of the Center ares (1) to advance the 
science of evaluation through extracting general evaluation principles 
and through developing generalized evaluation designs; and (2) to assist 
local schools and the Ohio State Education Department In the design and 
implementation of evaluations* In establishing the Center, it was 
immediately clear that neither of these purposes could be accomplished 
unless both were attacked simultaneously* Therefore, our Center is now 
actively engaged with the Ohio State Department of Education and with 
several Ohio school districts in developing and implementing evaluation 
designs* Through this approach, we hope to assist local schools with 
their immediate evaluation problems and, in so doing, to have the 
opportunity to study the evaluation process in detail, as it occurs in 
the real world* 

Our basic approach includes a full range of cooperation between the 
local school and the Center* We assist the local school In writing the 

V 

evaluation design and the evaluative reports that are required by both 
the project staff - for making decisions about the on-going project - 
and by the state education department, for making annual reports to the 
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Federal government* The local school provides staff members to serve 
on a school -and-uni versi ty evaluation team* These staff members are 
responsible for collecting data throughout the project* The Center 
takes the responsibility for training the school personnel In evaluation, 
in organizing and processing data, and in selecting and/or constructing 
the needed instrumentation* 

This approach, In my opinion, has several advantages* Since 
neither the local school districts nor the universities now possess 
sufficient numbers of trained evaluators, cooperation of the type de- 
scribed above offers an opportunity to increase the evaluation compe- 
tencies of local school personnel* Evaluation specialists in universities 
can provide "on the job" training to these local school personnel. In 
turn, the local schools can receive expert assistance in conducting the 
evaluations that are required now and can have increased evaluation 
expertise on their own staffs after the projects are completed* Also 
such collaboration will afford the universities an opportunity to study 
the evaluation process in the context of ordinary circumstances. This 
should facilitate the production of generalized evaluation designs which 
meet the requirements of school situations* 

RECOMMENDATIONS 

I would now like to conclude this presentation by listing several 
recommendations for your cans* deration# I realize that these are untried, 
and in that respect flimsy* However, they may provide a handle for some 
of the tough problems with which we shall be dealing during the remainder 
of this conference. If so, more than half of my purpose this morning 
will have been achieved. Here, then, are my recommendations. 
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1. Specific state plans for evaluation should be established . 

These should be set up Immediately, and they should be sufficiently 
specific so that comparable data can be collected from similar projects. 
Statewide objectives and criteria should be determined. Instruments 
should be selected and/or developed for measuring these objectives. 

Projects should be identified and classified according to these objectives. 
Statewide pretest norms on these instruments should be developed. 

Finally, these instruments should be administered on a statewide basis. 

By such a method, the effectiveness of the Title I program in the state 
could be assessed, l/ithout a specific plan, however, there will be 
little likelihood of making any sense out of the statewide picture. 

Unless my current information is faulty, at most only a few states have 
thus far prepared such plans. 

One major objection to this suggestion is that statewide testings 
would be too time consuming. However, recent developments in measure- 
ment theory by Frederic ’.ord^ and others have shown that test norms can 
be developed with no student attempting more than a sample of the test 
questions in any one test. This technique would seem to me to have 
tremendous Implications for statewide evaluations of Title 1 programs. 
Another more difficult objection is that tests for measuring the many 
classes of objectives that are possible under the Act do not now exist. 
However, this leads to my next recommendation. 

^Frederic M. Lord, Estimating Norms By Item-Sampling, Educational 
and Psychological Measurement, Vol. XXII, No. 2, 1962 
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2. Universities, stateeducat ion departments, and local school 
districts should collaborate to provide e ffective evaluations. Such 
collaboration is obviously indicated If specific statewide plans are to 
be implemented* Direction must come from the state education depart- 
ments* Universities can assist by developing the needed Instrumentation, 
suggesting appropriate data analysis designs, and conducting computer 
data analyses* Of course, the local schools will plan and implement the 
basic data collections and analyses* it seems to me that all of these 
functions can be accomplished better in concert than Independently* 

3. The evaluation design should not constrain or_ Interfere with 
the project design; rather, the evalua tion design should contribute. t.Q 
ttfi continual Improvement of the project* While project designs will 
Initially be based on the best knowledge that is available to the pro- 
ject director, these designs should be amenable to change and Improve- 
ment as more is learned about the operations of the projects* It 
would therefore seem Inappropriate to employ rigorously controlled 
experimental designs for evaluating projects* Such designs usually 
require that a narrow, constant treatment effect be applied In an error- 
free, laboratory-1 Ike context* In the biological sciences, the Impor- 
tance of an error-free context for experiments is seen in the great 
lengths that are taken to produce and employ gnotoblotlc, or germ free, 
situations* In the social sciences, we see the same concern over error 
variance In the complicated statistical designs which have been developed 
In order to extract as much of the error variance as possible* It Is 
patent that Title l-type projects will jjo£ be developed in, or installed 
Into, anything like error-free situations* It will also not be possible 
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to hold constant or to account for all of the interacting variables 
through the use of statistical designs that are presently available. 
(Indeed, one of the Jobs for evaluation Is to determine what the 
Important variables are.) Further projects ultimately wi 11 have to 
be designed to cope with the circumstances of ordinary school opera- 
tions. Therefore, It would seem appropriate to employ evaluation 
designs which are non- Interventionist, and which periodically provide 
objective data upon which decisions about modifying the project design 
and procedures can be based. 

4. Both product and process eva luation designs are needed. 

The product evaluations would be used to determine the overall effective- 
ness of the project, !.e., the extent to which specified standards have 
been reached. Such evaluations would provide the outcome data that will 
be required by the state and Federal governmental agencies. Probably 
the best non- Interventionist types of product evaluation designs are 
the quasi -experimental designs that have been suggested by Campbell and 
Stanley and others 

The process evaluations would provide information useful to the 
project staff for '(a) detecting any deviation of the actual project-from- 
the. project design, (b) improving the en-qoiog project, (c) Identifying 
possible problem areas In advance, (d) determining the strengths and 
weaknesses of the project design and procedures, (e) refining and 

^Donald T. Campbell and .Julian C. Stanley, "Experimental and Quasi- 
Experi mental Designs for Research on Teaching," N. L. Gage, editor, 
Handbook of Research on Teach 1 no « Ch. 3* Pp* 171*246. 

7 Gene V. Glass, "Evaluating Testing, Maturation, and Treatment - 
Effects in a Pretest-Posttest Quasi -Experimental Design, American 
Educational Research Journal > Voi. 2, March -1965, Pp. 83~82. 
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expanding the project design, (f) coordinating the over-all activities 
of the project schools, (g) facilitating communication among the pro- 
ject personnel, (h) diagnosing expected and unexpected outcomes of the 
project, and (1) communicating characteristics of the project to in- 
terested parties. The basic design for conducting process evaluations 
would be to employ resident evaluation specialists. These persons 
would be responsible for collecting process data on a day-to-day basis. 
Barker** has recently described this approach. The process evaluator 
would use such techniques as Interviews, classroom interaction analysis, 
PERT (Program Evaluation and Review Technique} Q-sort instruments, 
questionnaires, socio-drama, observation, and diaries. Medley and 
Mltzel^ have summarized and discussed a number of such techniques. 

5, Data collections should be objective:, therefore. It would seem 
appropriate to employ a special e valuation staff which is. independent, 
of the project staff . At the level of data collection, information 
should be collected as objectively as possible, and the information 
should not be tempered by the judgments of the evaluator toward that 
which is being evaluated. The information should essentially be raw, 
untampered-wi th data. Judgments, or evaluation, would seem to be 
appropriate only after a full set of data had been pulled together, 

This requirement would ufil seem to present a problem in the area of 
product evaluations. Such evaluations will essentially employ objective— 



^Roger G. Barker, "Explorations In Ecological Psychology," 

American Psvcholoolst . 20 (1965), 1-1^» 

^Donald M. Medley and Harold E. Mitzel , 'heasurlng Classroom Behavior 
by Systematic Observation," N. U Gage, editor, Handbook of Research on 
Teaching . Ch. 6 , Pp. 247-329* 
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type instruments, such as standardized tests, A problem would seem to 
exist, however, in the case of process evaluations where we will have 
to contend with the potential problem of human bias, since the essential 
instrument for obtaining process data is the resident evaluator. It 
would thus seem desirable that the resident evaluator jjgt have vested 
interests in the project, and that he be trained in the effective 
employment of observational data collection techniques. For these 
reasons, I would suggest that the process evaluations should not be 
conducted by the project staff, that instead a special evaluation staff 
be set up, trained, and employed to do this job. It would be desirable, 
however, that the members of this evaluation staff be drawn from the 
local school system. In this way the school could increase Its evalua- 
tlon competencies, through what I have described above as "on the job 
training and experience," for their staff, 

6, Schools should invest adequate resources Into, the design and 
Implementation of evaluations. If these are to become effective instru- 
ments for improving education . Title I projects will vary so widely, 
and my own experience is so limited, that I will not attempt to recommend 
dollar amounts for conducting evaluations, I do feel certain, however, 
that It will be necessary to invest more resources into evaluation than 
we have ever done in the past. To invest less than is required for pro- 
ducing sound, usable data would not seem wise* If there is one thing 
that educators can learn from industry, it is that wise investments in 
evaluation can pay off in progress. 
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CLOSING 



In closing, I certainly have no Illusions that I have solved any 
of the problems for which we are convened. I welcome your comments and 
criticisms and look forward to our association during the remainder of 
the conference. It has been a real pleasure to be here this morning, 
and I thank you for your attention. 





